# v1.3.0

# 基准测试

+ 使用[sofaload](https://github.com/antJack/sofaload)在本地搭建简单的性能测试

## v1.3.0测试结果

```Bash
sofaload -D 10 --qps=2000 -c 200 -t 16 -p sofarpc sofarpc://127.0.0.1:12200
starting benchmark...
Application protocol: sofarpc

finished in 10.00s, 1999.30 req/s, 2.41MB/s
requests: 20000 total, 20000 started, 19993 done, 19993 succeeded, 0 failed, 0 errored, 0 timeout
sofaRPC status codes: 
	19993 success, 0 error, 0 server exception, 0 unknown
	0 server threadpool busy, 0 error comm, 0 no processor, 0 timeout
	0 client send error, 0 codec exception, 0 connection closed, 0 server serial exception
	0 server deserial exception
traffic: 24.10MB (25271152) total, 390.49KB (399860) headers (space savings 0.00%), 23.72MB (24871292) data
                     min         max         mean         sd        +/- sd
time for request:      131us      7.67ms       273us       283us    96.33%
time for connect:        4us        36us        15us         6us    67.00%
req/s           :       9.50       10.60       10.00        0.41    80.50%

  Latency  Distribution
   50%        215us
   75%        288us
   90%        403us
   95%        504us
   99%       1.02ms
```

## v1.2.0测试结果

```Bash
sofaload -D 10 --qps=2000 -c 200 -t 16 -p sofarpc sofarpc://127.0.0.1:12200
starting benchmark...
Application protocol: sofarpc

finished in 10.00s, 2000.00 req/s, 2.41MB/s
requests: 20000 total, 20000 started, 20000 done, 20000 succeeded, 0 failed, 0 errored, 0 timeout
sofaRPC status codes: 
	20000 success, 0 error, 0 server exception, 0 unknown
	0 server threadpool busy, 0 error comm, 0 no processor, 0 timeout
	0 client send error, 0 codec exception, 0 connection closed, 0 server serial exception
	0 server deserial exception
traffic: 24.11MB (25280000) total, 390.63KB (400000) headers (space savings 0.00%), 23.73MB (24880000) data
                     min         max         mean         sd        +/- sd
time for request:      132us      8.59ms       289us       361us    97.33%
time for connect:        4us        87us        17us        17us    93.50%
req/s           :       9.50       10.60       10.00        0.41    77.00%

  Latency  Distribution
   50%        218us
   75%        296us
   90%        420us
   95%        533us
   99%       1.76ms
```

## 修复平滑升级失败

bug 描述：连续平滑升级仅第一次成功，后续均会失败

### before

#### 终端 1
```Bash
./mosn start -c mosn_config.json
2022-11-30 09:50:56,625 [INFO] register a new handler maker, name is default, is default: true
2022-11-30 09:50:56,629 [INFO] [admin server] [register api] register a new api /server_info
......
2022-11-30 09:50:56,640 [INFO] [admin store] [start service] start service Mosn Admin Server on [::]:34901
2022-11-30 09:50:57,641 [INFO] [server] [reconfigure] reconfigureHandler start
```

#### 终端 2
```Bash
./mosn start -c mosn_config.json
2022-11-30 09:53:58,264 [INFO] register a new handler maker, name is default, is default: true
2022-11-30 09:53:58,270 [INFO] [admin server] [register api] register a new api /server_info
......
2022-11-30 09:53:58,287 [INFO] [network] [transfer] [server] TransferServer start
2022-11-30 09:53:59,287 [INFO] [server] [reconfigure] reconfigureHandler start
2022-11-30 09:55:38,285 [INFO] [network] [transfer] [server] TransferServer exit
2022-11-30 09:55:38,287 [INFO] [network] [transfer] [server] TransferServer listener /Users/root/mosn/build/bundles/v1.2.0/binary/conn.sock closed
```

#### 终端 3
```Bash
2022-11-30 10:00:40,732 [INFO] register a new handler maker, name is default, is default: true
2022-11-30 10:00:40,739 [INFO] [admin server] [register api] register a new api /server_info
......
2022-11-30 10:00:40,758 [ERROR] [mosn] [NewMosn] start service failed: listen tcp 0.0.0.0:34901: bind: address already in use, exit
2022-11-30 10:00:40,758 [INFO] [mosn start] mosn clean upgrade datas
2022-11-30 10:00:40,758 [INFO] [mosn close] mosn stop server
2022-11-30 10:00:40,758 [FATAL] [network] [listener start] [listen] serverListener listen failed, listen tcp 127.0.0.1:2046: bind: address already in use
2022-11-30 10:00:40,758 [INFO] [admin store] [add service] add server Mosn Admin Server
2022-11-30 10:00:40,758 [INFO] [stagemanager] state changed to 9
```

### after

#### 终端 1
```Bash
./mosn start -c mosn_config.json
2022-11-30 10:18:43,248 [INFO] register a new handler maker, name is default, is default: true
2022-11-30 10:18:43,256 [INFO] [admin server] [register api] register a new api /server_info
......
2022-11-30 10:18:43,271 [INFO] [admin store] [start service] start service Mosn Admin Server on [::]:34901
2022-11-30 10:18:44,272 [INFO] [server] [reconfigure] reconfigureHandler start
```

#### 终端 2
```Bash
./mosn start -c mosn_config.json
2022-11-30 10:19:36,881 [INFO] register a new handler maker, name is default, is default: true
2022-11-30 10:19:36,889 [INFO] [admin server] [register api] register a new api /server_info
......
2022-11-30 10:19:36,908 [INFO] [network] [transfer] [server] TransferServer start
2022-11-30 10:19:37,909 [INFO] [server] [reconfigure] reconfigureHandler start
2022-11-30 10:21:16,912 [INFO] [network] [transfer] [server] TransferServer exit
2022-11-30 10:21:16,912 [INFO] [network] [transfer] [server] TransferServer listener /Users/root/mosn/build/bundles/v1.3.0/binary/conn.sock closed
```

#### 终端 3
```Bash
./mosn start -c mosn_config.json
2022-11-30 10:21:48,539 [INFO] register a new handler maker, name is default, is default: true
2022-11-30 10:21:48,546 [INFO] [admin server] [register api] register a new api /server_info
......
2022-11-30 10:21:48,562 [INFO] [network] [transfer] [server] TransferServer start
2022-11-30 10:21:49,563 [INFO] [server] [reconfigure] reconfigureHandler start
2022-11-30 10:23:28,566 [INFO] [network] [transfer] [server] TransferServer exit
2022-11-30 10:23:28,566 [INFO] [network] [transfer] [server] TransferServer listener /Users/root/mosn/build/bundles/v1.3.0/binary/conn.sock closed
```

## 修复 LB_ORIGINAL_DST LB 算法

bug 描述：cluster LB 算法为 LB_ORIGINAL_DST 时，依然需要配置至少一个 host (但实际不会用到) 才能正常使用

### Config
```json
{
    "servers": [
        {
            "default_log_path": "stdout",
            "listeners": [
                {
                    "name": "proxyListener",
                    "address": "0.0.0.0:12345",
                    "bind_port": true,
                    "log_path": "stdout",
                    "filter_chains": [
                        {
                            "tls_context": {},
                            "filters": [
                                {
                                    "type": "tcp_proxy",
                                    "config": {
                                        "cluster": "serverCluster",
                                    }
                                }
                            ]
                        }
                    ]
                }
            ]
        }
    ],
    "cluster_manager": {
        "clusters": [
            {
                "Name": "serverCluster",
                "type": "SIMPLE",
                "lb_type": "LB_ORIGINAL_DST",
                "max_request_per_conn": 1024,
                "conn_buffer_limit_bytes": 32768,
                "mark": 68
            }
        ]
    }
}
```

### server.go
```go
package main

import (
	"fmt"
	"net/http"
)

func handle(res http.ResponseWriter, req *http.Request) {
	fmt.Fprintln(res, "Hello Word")
}

func main() {
	addr := "0.0.0.0:8090"
	server := http.Server{
		Addr: addr,
	}

	http.HandleFunc("/", handle)
	server.ListenAndServe()
}
```

### 配置透明代理所需的 iptables 以及路由表，代理使用端口 12345，被代理端口 8090

```Bash
ip rule add fwmark 1 table 100
ip route add local 0.0.0.0/0 dev lo table 100
iptables -t mangle -I PREROUTING ! -d localhost -m mark --mark 0 -p tcp --dport 8090 -j TPROXY --on-port 12345 --tproxy-mark 1
```

### 启动 mosn 以及监听 8090 的 go 服务
```
./mosn start -c mosn_config.json
go run server.go
```

### 外部主机使用 curl 访问本机 8090 端口

#### before

```bash
curl 172.17.0.2:8090
curl: (56) Recv failure: Connection reset by peer
```

#### after
```bash
curl 172.17.0.2:8090
Hello Word
```

### 测试完成后删除相应配置

```Bash
ip rule del from all fwmark 1 lookup 100
ip route del local 0.0.0.0/0 dev lo table 100
iptables -t mangle -D PREROUTING ! -d localhost -m mark --mark 0 -p tcp --dport 8090 -j TPROXY --on-port 12345 --tproxy-mark 1
```

## 修改解析 xDS Listener 日志的方式 

参考[istio proxyv2 镜像构建](https://mosn.io/docs/user-guide/start/images/) 制作镜像
参考[集成 istio 示例](https://mosn.io/docs/user-guide/start/istio/) 搭建环境

### 进入 minikube 容器，随后使用 nsenter 进入一个 mosn 容器的网络命名空间，查看其配置
```Bash
docker exec -it minikube /bin/bash
root@minikube: container_pid=$(docker inspect --format '{{ .State.Pid }}' k8s_istio-proxy_reviews-v2-6d8fc985f7-l7lpg_default_013d7e34-a755-4aa8-acdb-ef0d9a569321_1)
root@minikube: nsenter -n -t$container_pid
root@minikube: curl 127.0.0.1:15000/api/v1/config_dump
```

#### before

```json
{
    "...": "...",
    "listener": {
        "...": "...",
        "virtualInbound": {
            "...": "...",
            "name": "virtualInbound",
            "type": "ingress",
            "address": "0.0.0.0:15006",
            "bind_port": true,
            "use_original_dst": "redirect",
            "access_logs": [
                {
                    "log_path": "/dev/stdout"
                },
                {
                    "log_path": "/dev/stdout"
                },
                {
                    "log_path": "/dev/stdout"
                },
                {
                    "log_path": "/dev/stdout"
                },
                {
                    "log_path": "/dev/stdout"
                },
                {
                    "log_path": "/dev/stdout"
                },
                {
                    "log_path": "/dev/stdout"
                }
            ]
        }
    }
}
```

#### after

```json
{
    "...": "...",
    "listener": {
        "...": "...",
        "virtualInbound": {
            "...": "...",
            "name": "virtualInbound",
            "type": "ingress",
            "address": "0.0.0.0:15006",
            "bind_port": true,
            "use_original_dst": "redirect",
            "access_logs": [
                {
                    "log_path": "/dev/stdout"
                }
            ]
        }
    }
}
```